Display device and operation method for same

ABSTRACT

According to an embodiment of the present disclosure, a display device includes a display, a communication interface configured to communicate with an external device, and a processor configured to receive language usage information of a user, determine a main language of the user based on the language usage information, compare a set system language with the main language, and when the system language is different from the main language, change the system language such that the system language is set to the main language or perform suggestion that the system language is set to the main language.

TECHNICAL FIELD

The present disclosure relates to a display device for updating a setsystem language and an operating method thereof.

BACKGROUND ART

Each user has his or her preferred and most used language, which may bedifferent. Accordingly, system languages preferred by users may bedifferent in a display device used by multiple users, such as a displaydevice installed in a building lobby or a display device in a guest roomof a lodging facility.

Conventionally, each user has to manually set the system language of thedisplay device when the system language set in the display device is nothis/her preferred language. When the user has to manually set the systemlanguage, it may cause great inconvenience to the user. In particular,when the previously set system language is a language that the user doesnot know, there is a problem in that it is difficult to modify thesystem language.

DISCLOSURE TECHNICAL PROBLEM

An object of the present disclosure is to provide a display device forupdating a set system language by determining a system language suitablefor a current user and automatically changing the set system language orsuggesting a change of a system language to a user, and an operatingmethod thereof.

Another object of the present disclosure is to provide a display devicefor determining whether to automatically change a system language afterdetermining a current user’s main language, and an operating methodthereof.

TECHNICAL SOLUTION

According to an embodiment of the present disclosure, a display deviceincludes a display, a communication interface configured to communicatewith an external device, and a processor configured to receive languageusage information of a user, determine a main language of the user basedon the language usage information, compare a set system language withthe main language, and when the system language is different from themain language, change the system language such that the system languageis set to the main language or perform suggestion that the systemlanguage is set to the main language.

The language usage information may include at least one of acontent-language entity of an HTTP response received from a website,language information of text included in a web page, languageinformation of voice of media content, language information of subtitlesof the media content, language information of the user’s spoken voice,and system language information of a user terminal.

The processor may select language usage information for a predeterminedperiod from among the language usage information, assign a weight toeach of the selected language usage information, and determine the mainlanguage in consideration of the weight corresponding to each of theselected language usage information.

The processor may assign a higher time weight to each of the selectedlanguage usage information as an elapsed time is shorter, assign apredetermined type weight to each of the selected language usageinformation for each type of language usage information, and calculatean overall weight based on the time weight and the type weight.

The processor may calculate a sum of the overall weights for eachlanguage based on the overall weight calculated for each of the selectedlanguage usage information, and determine a language having a largestsum of the overall weights as the main language.

The processor may determine the main language in consideration of aweight corresponding to the selected language usage information and apredetermined priority for each type of language usage information.

The processor may select a first language usage information having ahighest priority from among the selected language usage information, anddetermine the main language in consideration of a weight of the firstlanguage usage information.

A priority of the system language of the user terminal may be higherthan a priority of other language usage information, and a priority ofthe language of the subtitles of the media content may be higher than apriority of the language of the voice of the media content.

The processor may select language usage information for a predeterminedperiod from among the language usage information, and determine the mainlanguage using the selected language usage information and a mainlanguage determination model, and the main language determination modelmay be a model including an artificial neural network (ANN), the modelbeing trained based on a machine learning algorithm or a deep learningalgorithm.

The main language determination model may be a model that includes aRecurrent Neural Network (RNN), and determines and outputs the mainlanguage when the language usage information is input in real time.

The processor may generate learning data corresponding to information ona used language and a result of change when changing or attempting tochange the system language, and the learning data may be used to updatethe main language determination model.

The processor may automatically change the system language such that thesystem language is set to the main language when the system language andthe main language are different from each other, output a firstnotification indicating that the system language has been changed, anddetermine whether to cancel change of the system language based on aninput of the user corresponding to the first notification in a casewhere a language update mode is an automatic update mode.

The processor may output a second notification suggesting change of thesystem language such that the system language is set to the mainlanguage when the system language and the main language are differentfrom each other, output a first notification indicating that the systemlanguage has been changed, and determine whether to change the systemlanguage based on an input of the user corresponding to the secondnotification in a case where the language update mode is a manual updatemode.

According to an embodiment, a method for operating a display deviceincludes receiving language usage information of a user, determining amain language of the user based on the language usage information,comparing a set system language with the main language, and changing thesystem language such that the system language is set to the mainlanguage or performing suggestion that the system language is set to themain language, when the system language is different from the mainlanguage.

According to an embodiment, a recording medium recording a method ofoperating a display device includes receiving language usage informationof a user, determining a main language of the user based on the languageusage information, comparing a set system language with the mainlanguage, and changing the system language such that the system languageis set to the main language or performing suggestion that the systemlanguage is set to the main language, when the system language isdifferent from the main language.

ADVANTAGEOUS EFFECTS

According to various embodiments of the present disclosure, it ispossible to easily update system language settings even if a user doesnot select a system language to be updated by determining a systemlanguage suitable for a current user, thus increasing user convenience.

Further, according to various embodiments of the present disclosure, itis possible to determine whether to automatically update system languagesettings based on a user’s usage pattern, thus providing a systemlanguage setting function with high satisfaction for each user.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a display device according to anembodiment of the present disclosure.

FIG. 2 is a block diagram illustrating a remote control device accordingto an embodiment of the present disclosure.

FIG. 3 is a diagram illustrating a remote control device according to anembodiment of the present disclosure.

FIG. 4 shows an example of interworking with a display device through aremote control device according to an embodiment of the presentdisclosure.

FIG. 5 is a block diagram illustrating an artificial intelligence serveraccording to an embodiment of the present disclosure.

FIG. 6 is a flowchart showing a method of operating a display deviceaccording to an embodiment of the present disclosure.

FIG. 7 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

FIG. 8 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

FIG. 9 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

FIG. 10 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

FIG. 11 is a diagram illustrating an example of weights and prioritiesassigned to types of language usage information.

FIG. 12 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

FIG. 13 is a diagram illustrating an example of a main languagedetermination model according to an embodiment of the presentdisclosure.

FIG. 14 is a diagram illustrating an embodiment of the presentdisclosure.

FIG. 15 is a diagram illustrating an embodiment of the presentdisclosure.

FIG. 16 is a diagram illustrating an embodiment of the presentdisclosure.

FIG. 17 is a diagram illustrating an embodiment of the presentdisclosure.

MODE FOR INVENTION

Hereinafter, the embodiments disclosed herein will be described indetail with reference to the accompanying drawings, and the same orsimilar elements are designated with the same numeral referencesregardless of the numerals in the drawings and their redundantdescription will be omitted. The suffixes “module” and “unit or portion”for components used in the following description are merely providedonly for facilitation of preparing this specification, and thus they arenot granted a specific meaning or function. In addition, when it isdetermined that the detailed description of the related known technologymay obscure the gist of embodiments disclosed herein in describing theembodiments, a detailed description thereof will be omitted. Further,the accompanying drawings are intended to facilitate understanding ofthe embodiments disclosed herein, and the technical spirit disclosedherein are not limited by the accompanying drawings. Therefore, thepresent disclosure should be construed as including all the changes,equivalents, and substitutions included in the spirit and scope of thepresent disclosure.

The terms coming with ordinal numbers such as ‘first’, ‘second’, or thelike may be used to denote various components, but the components arenot limited by the terms. The terms are used merely for the purpose todistinguish a component from the other component.

It will also be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present.

FIG. 1 is a block diagram showing a configuration of a display deviceaccording to an embodiment of the present disclosure.

Referring to FIG. 1 , the display device 100 may be connected to atleast one of a remote control device 200, a terminal 300, and anartificial intelligence server 400 to transmit/receive data or signals.

The display device 100 refers to a device capable of outputting an imageincluding a display 180 or a display panel, and may be implemented by astationary device or a mobile device, such as a TV, a projector, amobile phone, a smartphone, a desktop computer, a laptop, a digitalbroadcasting terminal, a personal digital assistant (PDA), a portablemultimedia player (PMP), a navigation device, a tablet PC, a wearabledevice, a set-top box (STB), a DMB receiver, a radio, a washing machine,a refrigerator, a digital signage, a robot, a vehicle, and the like.

The user terminal 300 may be implemented as a mobile phone, a smartphone, a tablet PC, a laptop, a wearable device, a PDA, or the like.

The display device 100 may include a communication interface 110, abroadcast receiver 130, an external device interface 135, a memory 140,an input interface 150, a processor 170, a display 180, a speaker 185,and a power supply circuit 190.

The communication interface 110 may perform a wired or wirelesscommunication with an external electronic device. For example, thecommunication interface 110 may transmit and receive sensor information,a user input, a learning model, and a control signal to and fromexternal devices, such as another display device. Here, the anotherdisplay device 100 may be a wearable device (e.g., a smartwatch, smartglasses or a head mounted display (HMD), a mobile terminal such as asmart phone, which is able to exchange data (or interwork) with thedisplay device 100 according to the present disclosure.

The communication interface 110 may detect (or recognize) a wearabledevice capable of communication around the display device 100.Furthermore, if the detected wearable device is a device authenticatedto communicate with the display device 100, the processor 170 cantransmit at least part of data processed in the display device 100 tothe wearable device through the communication interface 110. Therefore,a user of the wearable device may use data processed by the displaydevice 100 through the wearable device.

The communication technology used by the communication interface 110includes GSM (Global System for Mobile communication), CDMA (CodeDivision Multi Access), LTE (Long Term Evolution), 5G, WLAN (WirelessLAN), Wi-Fi (Wireless-Fidelity), Bluetooth™, RFID (Radio FrequencyIdentification), Infrared Data Association (IrDA), ZigBee, NFC (NearField Communication), and the like.

The communication interface 110 may also be referred to as acommunication modem or a communication interface.

The broadcast receiver 130 may include a tuner 131, a demodulator 132,and a network interface 133.

The tuner 131 may select a specific broadcast channel according to achannel selection command. The tuner 131 may receive a broadcast signalfor the selected specific broadcast channel.

The demodulator 132 may separate the received broadcast signal into animage signal, an audio signal, and a data signal related to a broadcastprogram, and restore the separated image signal, audio signal, and datasignal to a format capable of being output.

The external device interface 135 may receive an application or a listof applications in an external device adjacent thereto, and transmit thesame to the controller 170 or the memory 140.

The external device interface 135 may provide a connection path betweenthe display device 100 and an external device. The external deviceinterface 135 may receive one or more of images and audio output from anexternal device connected to the display device 100 in a wired orwireless manner, and transmit the same to the controller 170. Theexternal device interface 135 may include a plurality of external inputterminals. The plurality of external input terminals may include an RGBterminal, one or more High Definition Multimedia Interface (HDMI)terminals, and a component terminal.

The image signal of the external device input through the externaldevice interface 135 may be output through the display 180. The audiosignal of the external device input through the external deviceinterface 135 may be output through the speaker 185.

The external device connectable to the external device interface 135 maybe any one of a set-top box, a Blu-ray player, a DVD player, a gamemachine, a sound bar, a smartphone, a PC, a USB memory, and a hometheater, but this is only an example.

The network interface 133 may provide an interface for connecting thedisplay device 100 to a wired/wireless network including an Internetnetwork. The network interface 133 may transmit or receive data to orfrom other users or other electronic devices through a connected networkor another network linked to the connected network.

In addition, a part of content data stored in the display device 100 maybe transmitted to a selected user among a selected user or a selectedelectronic device among other users or other electronic devicesregistered in advance in the display device 100.

The network interface 133 may access a predetermined web page throughthe connected network or the other network linked to the connectednetwork. That is, it is possible to access a predetermined web pagethrough a network, and transmit or receive data to or from acorresponding server.

In addition, the network interface 133 may receive content or dataprovided by a content provider or a network operator. That is, thenetwork interface 133 may receive content such as movies,advertisements, games, VOD, and broadcast signals and informationrelated thereto provided from a content provider or a network providerthrough a network.

In addition, the network interface 133 may receive update informationand update files of firmware provided by the network operator, and maytransmit data to an Internet or content provider or a network operator.

The network interface 133 may select and receive a desired applicationfrom among applications that are open to the public through a network.

The memory 140 may store programs for signal processing and control ofthe controller 170, and may store images, audio, or data signals, whichhave been subjected to signal-processed. For example, the memory 170 maystore input data acquired by the input unit 120, learning data, alearning model, a learning history, and the like.

The memory 140 may perform a function for temporarily storing video,audio, or data signals input from an external device interface 135 orthe network interface unit 133, and store information on a predeterminedimage through a channel storage function.

The memory 140 can store an application or an application list inputfrom the external device interface 135 or the network interface unit133.

The display device 100 may play back a content file (a moving imagefile, a still image file, a music file, a document file, an applicationfile, or the like) stored in the memory 140 and provide the same to theuser.

The input interface 150 may acquire various kinds of data. In this case,the input interface 120 may include a camera for inputting a videosignal, a microphone for receiving an audio signal, and a user inputunit for receiving information from a user.

The user input interface may transfer signals input by a user to theprocessor 170 or transfer signals from the processor170 to a user. Forexample, the user input interface unit 150 may receive and process acontrol signal such as power on/off, channel selection, screen settings,and the like from the remote control device 200 in accordance withvarious communication methods, such as a Bluetooth communication method,a WB (Ultra Wideband) communication method, a ZigBee communicationmethod, an RF (Radio Frequency) communication method, or an infrared(IR) communication method or may perform processing to transmit thecontrol signal from the processor 170 to the remote control device 200.

Additionally, the user input interface 150 may transfer, to theprocessor 170, control signals input from local keys (not shown) such asa power key, a channel key, a volume key, and a setting value.

The learning processor 160 may learn a model composed of an artificialneural network by using learning data. The learned artificial neuralnetwork may be referred to as a learning model. The learning model maybe used to an infer result value for new input data rather than learningdata, and the inferred value may be used as a basis for determination toperform a certain operation.

In this case, the learning processor 160 may perform AI processingtogether with the learning processor 440 of the AI server 400.

At this time, the learning processor 160 may include a memory integratedor implemented in the display device 100. Alternatively, the learningprocessor 160 may be implemented by using the memory 170, an externalmemory directly connected to the display device 100, or a memory held inan external device.

Image signals that are image-processed in the processor 170 can be inputto the display 180 and displayed as an image corresponding tocorresponding image signals. Additionally, the image signals that areimage-processed in the processor 170 can be input to an external outputdevice through an external device interface 135.

The audio signal processed by the processor 170 may be output to thespeaker 185. Also, the audio signal processed by the processor 170 maybe input to the external output device through the external deviceinterface 135.

The processor 170 may control overall operation of the recognitiondevice 100.

Additionally, the processor 170 can control the display device 100 by auser command or internal program input through the user input interfaceand download a desired application or application list into the displaydevice 100 in access to network.

The processor 170 may output channel information selected by a usertogether with processed image or voice signals through the display 180or the speaker 185.

Additionally, according to an external device image playback commandreceived through the user interface, the processor 170 can output imagesignals or voice signals of an external device such as a camera or acamcorder, which are input through the external device interface 135,through the display 180 or the speaker 185.

Moreover, the processor 170 can control the display 180 to displayimages and control broadcast images input through the tuner 131,external input images input through the external device interface 135,images input through the network interface unit, or images stored in thememory 140 to be displayed on the display 180. In this case, an imagedisplayed on the display 180 can be a still image or video and also canbe a 2D image or a 3D image.

Additionally, the processor 170 can play content stored in the displaydevice 100, received broadcast content, and external input content inputfrom the outside, and the content can be in various formats such asbroadcast images, external input images, audio files, still images,accessed web screens, and document files.

The processor 170 may determine at least one executable operation of thedisplay device 100 based on information determined or generated by usinga data analysis algorithm or a machine learning algorithm. The processor170 may control the components of the display device 100 to execute thedetermined operation.

To this end, the processor 170 may request, search, receive, or utilizedata of the learning processor 160 or the memory 140. The processor 180may control the components of the display device 100 to execute thepredicted operation or the operation determined to be desirable amongthe at least one executable operation.

The processor 170 may acquire intention information for the user inputand may determine the user’s requirements based on the acquiredintention information.

The processor 170 may acquire the intention information corresponding tothe user input by using at least one of a speech to text (STT) enginefor converting speech input into a text string or a natural languageprocessing (NLP) engine for acquiring intention information of a naturallanguage.

At least one of the STT engine or the NLP engine may be configured as anartificial neural network, at least part of which is learned accordingto the machine learning algorithm. At least one of the STT engine or theNLP engine may be learned by the learning processor 160, may be learnedby the learning processor 440 of an artificial intelligence server 400,or may be learned by their distributed processing.

The processor 170 may collect history information including theoperation contents of the display device 100 or the user’s feedback onthe operation and may store the collected history information in thememory 150 or the learning processor 160 or transmit the collectedhistory information to the external device such as the AI server 400.The collected history information may be used to update the learningmodel.

The display 180 may convert image signals, data signals, and OSD signalsprocessed by the processor 170, or image signals or data signalsreceived from the external device interface 135 into R, G, and B signalsto output images.

Meanwhile, the display device 100 illustrated in FIG. 1 is merely anembodiment of the present disclosure, and some of the illustratedcomponents may be integrated, added, or omitted according to thespecifications of the display device 100 that are actually implemented.

In an embodiment, two or more components of the display device 100 maybe combined into one component, or one component may be divided into twoor more components as necessary. In addition, a function performed ineach block is for describing an embodiment of the present disclosure,and its specific operation or device does not limit the scope of thepresent disclosure.

According to an embodiment of the present disclosure, unlike the displaydevice 100 shown in FIG. 1 , the display device 100 may receive an imagethrough the network interface unit 133 or the external device interface135 without a tuner 131 and a demodulator 132 and play back the same.For example, the display device 100 may be divided into an imageprocessing device, such as a set-top box, for receiving broadcastsignals or content according to various network services, and a contentplayback device that plays back content input from the image processingdevice. In this case, an operation method of the display deviceaccording to an embodiment of the present disclosure will be describedbelow may be implemented by not only the display device 100 as describedwith reference to FIG. 1 and but also one of an image processing devicesuch as the separated set-top box and a content playback deviceincluding the display 180 the speaker 185.

FIG. 2 is a block diagram illustrating a remote control device 200according to an embodiment of the present disclosure.

First, referring to FIG. 2 , the remote control device 200 may include afingerprint reader 210, a wireless communication circuit 220, a userinput interface 230, a sensor 240, an output interface 250, a powersupply circuit 260, a memory 270, a controller 280, and a microphone290.

Referring to FIG. 2 , the wireless communication circuit 220 maytransmit and receive signals to and from any one of display devicesaccording to embodiments of the present disclosure described above.

The remote control device 200 may include an RF circuit 221 capable oftransmitting and receiving signals to and from the display device 100according to the RF communication standard, and an IR circuit 223capable of transmitting and receiving signals to and from the displaydevice 100 according to the IR communication standard. In addition, theremote control device 200 may include a Bluetooth circuit 225 capable oftransmitting and receiving signals to and from the display device 100according to the Bluetooth communication standard. In addition, theremote control device 200 may include an NFC circuit 227 capable oftransmitting and receiving signals to and from the display device 100according to the NFC (near field communication) communication standard,and a WLAN circuit 229 capable of transmitting and receiving signals toand from the display device 100 according to the wireless LAN (WLAN)communication standard.

In addition, the remote control device 200 may transmit a signalcontaining information on the movement of the remote control device 200to the display device 100 through the wireless communication circuit220.

In addition, the remote control device 200 may receive a signaltransmitted by the display device 100 through the RF circuit 221, andtransmit a command regarding power on/off, channel change, volumeadjustment, or the like to the display device 100 through the IR circuit223 as necessary.

The user input interface 230 may include a keypad, a button, a touchpad, a touch screen, or the like. The user may input a command relatedto the display device 100 to the remote control device 200 by operatingthe user input interface 230. When the user input interface 230 includesa hard key button, the user may input a command related to the displaydevice 100 to the remote control device 200 through a push operation ofthe hard key button.

When the user input interface 230 includes a touch screen, the user mayinput a command related to the display device 100 to the remote controldevice 200 by touching a soft key of the touch screen. In addition, theuser input interface 230 may include various types of input means thatmay be operated by a user, such as a scroll key or a jog key.

The sensor 240 may include a gyro sensor 241 or an acceleration sensor243, and the gyro sensor 241 may sense information regarding themovement of the remote control device 200. For example, the gyro sensor241 may sense information about the operation of the remote controldevice 200 based on the x, y, and z axes, and the acceleration sensor243 may sense information about the moving speed of the remote controldevice 200. Meanwhile, the remote control device 200 may further includea distance measuring sensor to sense the distance between the displaydevice 100 and the display 180.

The output interface 250 may output an image or audio signalcorresponding to the operation of the user input interface 230 or asignal transmitted from the display device 100. The user may recognizewhether the user input interface 230 is operated or whether the displaydevice 100 is controlled through the output interface 250. For example,the output interface 450 may include an LED 251 that emits light, avibrator 253 that generates vibration, a speaker 255 that outputs sound,or a display 257 that outputs an image when the user input interface 230is operated or a signal is transmitted and received to and from thedisplay device 100 through the wireless communication unit 225.

The power supply circuit 260 may supply power to the remote controldevice 200. The power supply unit 260 may stop power supply when theremote control device 200 has not moved for a predetermined time toreduce power consumption. The power supply circuit 260 may restart powersupply when a predetermined key provided in the remote control device200 is operated.

The memory 270 may store various types of programs and application datarequired for control or operation of the remote control device 200.

When the remote control device 200 transmits and receives signalswirelessly through the display device 100 and the RF circuit 221, theremote control device 200 and the display device 100 transmit andreceive signals through a predetermined frequency band. The controller280 of the remote control device 200 may store and refer to informationon a frequency band capable of wirelessly transmitting and receivingsignals to and from the display device 100 paired with the remotecontrol device 200 in the memory 270.

The controller 280 may control all matters related to the control of theremote control device 200. The controller 280 may transmit a signalcorresponding to a predetermined key operation of the user inputinterface 230 or a signal corresponding to the movement of the remotecontrol device 200 sensed by the sensor 240 through the wirelesscommunication unit 225.

The microphone 290 may obtain a speech. The microphone 290 may includeat least one microphone and obtain voice through the microphone 291.

FIG. 3 is a block diagram showing a remote control device 200 accordingto an embodiment of the present disclosure.

Referring to FIG. 3 , the remote control device 200 may include aplurality of buttons. The plurality of buttons may include a fingerprintrecognition button 212, a power button 231, a home button 232, a livebutton 233, an external input button 234, a volume control button 235, avoice recognition button 236, a channel change button 237, an OK button238, and a back-play button 239.

The fingerprint recognition button 212 may be a button for recognizing auser’s fingerprint. In one embodiment, the fingerprint recognitionbutton 212 may enable a push operation, and thus may receive a pushoperation and a fingerprint recognition operation. The power button 231may be a button for turning on/off the power of the display device 100.The home button 232 may be a button for moving to the home screen of thedisplay device 100. The live button 233 may be a button for displaying areal-time broadcast program. The external input button 234 may be abutton for receiving an external input connected to the display device100. The volume control button 235 may be a button for adjusting thelevel of the volume output by the display device 100. The voicerecognition button 236 may be a button for receiving a user’s voice andrecognizing the received voice. The channel change button 237 may be abutton for receiving a broadcast signal of a specific broadcast channel.The OK button 238 may be a button for selecting a specific function, andthe back-play button 239 may be a button for returning to a previousscreen.

FIG. 4 shows an example of interworking with the display device 100through a remote control device 200 according to an embodiment of thepresent disclosure.

Referring to FIG. 4 , a pointer 205 corresponding to the remote controldevice 200 may be displayed on the display 180.

Referring to (a) in FIG. 4 , the user may move or rotate the remotecontrol device 200 up, down, left and right.

The pointer 205 displayed on the display 180 of the display device 100may move in correspondence with the movement of the remote controldevice 200. The pointer 205 is moved and displayed according to movementof the remote control device 200 in a 3D space, so the remote controldevice 200 may be called a space remote control device.

Referring to (b) in FIG. 4 , when the user moves the remote controldevice 200 to the left, the pointer 205 displayed on the display 180 ofthe display device 100 may also move to the left correspondingly.

Information on the movement of the remote control device 200 detectedthrough a sensor of the remote control device 200 may be transmitted tothe display device 100. The display device 100 may calculate thecoordinates of the pointer 205 based on information on the movement ofthe remote control device 200 and display the pointer 205 to correspondto the calculated coordinates.

In (c) of FIG. 4 , it is illustrated that a user moves the remotecontrol device 200 away from the display 180 while pressing a specificbutton in the remote control device 200. Accordingly, a selected area inthe display 180 corresponding to the pointer 205 may be zoomed in anddisplayed enlarged. A selected area in the display 180 corresponding tothe pointer 205 may be zoomed in and displayed enlarged. Conversely,when the user moves the remote control device 200 to be close to thedisplay 180 while a specific button of the remote control device 200 isbeing pressed, the selected area in the display 180 corresponding to thepointer 205 may be zoomed out and displayed reduced.

On the other hand, when the remote control device 200 moves away fromthe display 180, the selected area may be zoomed out, and when theremote control device 200 moves to be close to the display 180, theselected area may be zoomed in.

Also, in a state in which a specific button in the remote control device200 is being pressed, recognition of up, down, left, or right movementsmay be excluded. That is, when the remote control device 200 moves awayfrom or close to the display 180, the up, down, left, or right movementsare not recognized, and only the forward and backward movements may berecognized. In a state in which a specific button in the remote controldevice 200 is not being pressed, only the pointer 205 moves according tothe up, down, left, or right movements of the remote control device 200.

Meanwhile, the movement speed or the movement direction of the pointer205 may correspond to the movement speed or the movement direction ofthe remote control device 200.

Meanwhile, in the present specification, a pointer refers to an objectdisplayed on the display 180 in response to an operation of the remotecontrol device 200. Accordingly, objects of various shapes other thanthe arrow shape shown in the drawings are possible as the pointer 205.For example, the object may be a concept including a dot, a cursor, aprompt, a thick outline, and the like. In addition, the pointer 205 maybe displayed corresponding to any one point among points on a horizontalaxis and a vertical axis on the display 180, and may also be displayedcorresponding to a plurality of points such as a line and a surface.

FIG. 5 is a block diagram illustrating an artificial intelligence (AI)server 400 according to an embodiment of the present disclosure.

Referring to FIG. 5 , the AI server 400 may refer to a device thatlearns an artificial neural network by using a machine learningalgorithm or uses a learned artificial neural network. The AI server 400may include a plurality of servers to perform distributed processing, ormay be defined as a 5G network.

The AI server 400 may perform at least part of the AI processing of thedisplay device 100 together. The artificial intelligence processing maymean an operation required for training of an artificial intelligencemodel.

The AI server 400 may include a communication device 410, a memory 430,a learning processor 440, a processor 460, and the like.

The communication device 410 can transmit and receive data to and froman external device such as the display device 100.

The memory 430 may include a model storage 431. The model storage 431may store a model 431 a (or an artificial neural network), which isbeing trained or has been trained, through the learning processor 440.

The learning processor 440 may learn the artificial neural network 431 aby using the learning data. The learning model may be used in a state ofbeing mounted on the AI server 400 of the artificial neural network, ormay be used in a state of being mounted on an external device such asthe display device 100.

The learning model may be implemented in hardware, software, or acombination of hardware and software. If all or some of the learningmodels are implemented in software, one or more instructions thatconstitute the learning model may be stored in memory 430.

The processor 460 may infer the result value for new input data by usingthe learning model and may generate a response or a control commandbased on the inferred result value.

FIG. 6 is a flowchart showing a method of operating a display device 100according to an embodiment of the present disclosure.

Referring to FIG. 6 , the processor 170 of the display device 100 mayreceive language usage information of a user (S601).

The language usage information of the user may include information aboutlanguage used by the user terminal 300 as well as information aboutlanguage used by the display device 100.

The language usage information of the user may include thecontent-language entity of an HTTP response received from a websiteduring a web browsing process, the language information of text includedin the web page, the language information of voice of media content,language information of subtitles of media content, language informationof a user’s spoken voice, system language information of the userterminal 300, and the like.

The content-language entity may be included in an HTTP response receivedfrom a website during a web browsing process through a web browserapplication of the display device 100. For example, when thecontent-language entity of the HTTP response received from a specificwebsite is “en-US”, this indicates that the language of the website isEnglish.

The language information of voice of media content may mean the languageinformation of a selected voice when the language of the voice of themedia content is selectable.

The language information of subtitle of media content may mean thelanguage information of a selected subtitle when the language of thesubtitle corresponding to the media content is selectable.

The user’s spoken voice may be received through the microphone of theinput interface 150 or may be received from the user terminal 300through the communication interface 110. The processor 170 may obtainthe language information of the user’s spoken voice by using a languagediscrimination model.

The language discrimination model may refer to a model that outputs alanguage corresponding to voice data when the voice data is input. Thelanguage discrimination model may include an artificial neural network(ANN), and may be trained using a machine learning algorithm or a deeplearning algorithm. The language discrimination model may be trained bythe processor 170 or the learning processor 160 of the display device100, or may be trained by the processor 460 or the learning processor440 of the AI server 400.

The language usage information may include time information. The timeinformation included in the language usage information may be used todetermine how old each language usage information is. In this regard,the system language information of the user terminal 300 may representreal-time information or may not include the real-time information. Thatis, the system language information may be state information.

Then, the processor 170 of the display device 100 may determine theuser’s main language based on the language usage information (S603).

The main language refers to a language mainly used by a user, and alanguage most frequently used may be determined as the main language.For example, when the user is browsing a web through a web browserapplication, and most of the visited web pages (e.g., 90% or more) areKorean web pages, the processor 170 may determine the user’s mainlanguage as Korean. For example, when the user’s spoken voice is Korean,the processor 170 may determine the user’s main language as Korean. Forexample, when the user sets the language of subtitle of media content toKorean, the processor 170 may determine the user’s main language asKorean. For example, when the display device 100 and the user terminal300 interwork with each other and the system language of the userterminal 300 is Korean, the processor 170 may determine the user’s mainlanguage as Korean.

However, since priorities or weights are different for pieces oflanguage usage information, even a language that is not frequently usedmay be determined as the main language. The processor 170 may determinethe user’s main language in consideration of the weight or prioritygiven to each language usage information with respect to the receivedlanguage usage information.

In addition, the processor 170 may determine the main language by givinga higher weight to the language usage information for which the elapsedtime is short in consideration of the time information of the receivedlanguage usage information.

In an embodiment, the content-language entity of the HTTP response mayhave a higher priority than the language information of text included ina web page. In an embodiment, the language information of subtitle ofthe media content may have a higher priority than the languageinformation of voice of media content. In an embodiment, the systemlanguage information of the user terminal 300 may have a higher prioritythan other language usage information.

Also, the processor 170 may determine the user’s main language by usingthe main language determination model.

The main language determination model may refer to a model fordetermining (or discriminating) the user’s main language from thelanguage usage information of the user. That is, the main languagedetermination model may be a model for outputting the main language of acorresponding user when the language usage information of the user isinput.

The main language determination model may refer to a model fordetermining whether to automatically update the system language suchthat the system language is set to the main language based on a machinelearning algorithm or a deep learning algorithm. The main languagedetermination model may include an artificial neural network (ANN).

The main language determination model may be trained by the processor170 or the learning processor 160 of the display device 100, or may betrained by the processor 460 or the learning processor 440 of the AIserver 400. Furthermore, the main language determination model may betrained not only by the processor 170 or the learning processor 160 ofthe display device 100, but also by the processor 460 or the learningprocessor 440 of the AI server 400.

Then, the processor 170 of the display device 100 may determine whetherthe system language and the main language are the same (S605).

The system language of the display device 100 may mean a language set inthe display device 100, and may include a language displayed in a menu,a language displayed in help, a language set in an application, a voiceassistant language, an audio guide language, and the like.

The system language of the display device 100 may include a system textlanguage and a system voice language. It is preferable that the systemtext language and the system voice language be set to be the same, but,generally, the system text language and the system voice language may beset differently in that the supported system text language is more thanthe supported system voice language.

When it is determined in step S605 that the system language and the mainlanguage are the same, the processor 170 of the display device 100 mayterminate the process without changing the set system language.

When it is determined in step S605 that the system language and the mainlanguage are different from each other, the processor 170 of the displaydevice 100 may determine whether a language update mode is an automaticupdate mode (S607).

The language update mode may mean a mode for setting whether toautomatically update the set system language when the system languageand the main language are different from each other. The processor 170may perform a process of updating the set system language based on theset language update mode.

When it is determined in step S607 that the set language update mode isthe automatic update mode, the processor 170 of the display device 100may automatically change the system language such that the systemlanguage is set to the main language (S609).

The automatic update mode may mean a mode for automatically updating thesystem language. Accordingly, when the set language update mode is theautomatic update mode, the processor 170 may update the system languagesuch that the system language is set to the determined main languagewithout asking the user whether to update the system language.

However, when the processor 170 automatically changes the systemlanguage without asking the user’s intention, a situation in which thesystem language change is against the user’s will may occur.Accordingly, in an embodiment, after automatically changing the systemlanguage, the processor 170 may output a notification indicating thatthe system language is automatically changed through the display 180,and the output notification may include an item to cancel the change ofthe system language.

When it is determined in step S607 that the set language update mode isnot the automatic update mode (that is, when the set language updatemode is the manual update mode), the processor 170 of the display device100 may output a notification suggesting update of the system languagethrough the display 180 (S611), and determine whether the user’sresponse is to accept the update (S613).

The manual update mode may mean a mode in which an update of the systemlanguage is suggested to the user and the system language is updatedwhen the user makes an agreement with the update. Accordingly, when theset language update mode is the manual update mode, the processor 170may suggest update of the system language to the suer, and determinewhether to update the system language based on the user’s response.

The notification suggesting update of the system language may include atleast one of information on a currently set system language, informationon a main language to be updated, an item accepting update, or an itemrefusing update.

The notification suggesting the update of the system language may beprovided in two languages: a currently set system language and a mainlanguage to be updated. For example, when the currently set systemlanguage is English and the main language to be updated is Korean, theprocessor 170 may output a notification suggesting change of the systemlanguage from English to Korean through the display 180 in both Englishand Korean or in a manner of alternating English and Korean.

When it is determined in step S613 that the user’s response is to acceptthe update of the system language, the processor 170 of the displaydevice 100 may change the system language such that the system languageis set to the main language (S615), and generate learning data from thesystem language update record (S617).

After changing the system language to the main language, the processor170 may output a notification indicating that the system language is setto the main language through the display 180.

When it is determined in step S613 that the user’s response is to refusethe update of the system language, the processor 170 of the displaydevice 100 may not change the system language such that the systemlanguage is set to the main language, and generate learning data fromthe system language update record (S617).

The learning data may include language usage information, a previouslyset system language, a main language, and whether the system language isupdated. The generated learning data may be used for training the mainlanguage determination model.

The processor 170 or the learning processor 160 of the display device100 may update the main language determination model by using thegenerated learning data, and store the updated main languagedetermination model in the memory 140. Alternatively, the processor 170of the display device 100 may transmit the generated learning datathrough the communication interface 110 to the AI server 400. Theprocessor 460 or the learning processor 440 of the AI server 400 mayupdate the main language determination model using the learning data,and the processor 170 of the display device 100 may receive the updatedmain language determination model from the AI server 400 through thecommunication interface 110 and store the updated main languagedetermination model in the memory 140.

The order of the steps shown in FIG. 6 is only an example, and thepresent disclosure is not limited thereto. That is, according to anembodiment, the order of some of the steps shown in FIG. 6 may bereversed to be performed. In an embodiment, some of the steps shown inFIG. 6 may be performed in parallel. Further, only some of the stepsshown in FIG. 6 may be performed.

FIG. 6 illustrates only one cycle of a method of operating the displaydevice 100, and the present disclosure is not limited thereto. That is,the steps shown in FIG. 6 may be repeatedly performed.

FIG. 7 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

Specifically, FIG. 7 illustrates an embodiment in which the displaydevice 100 determines a main language according to a rule-baseddetermination method in a situation in which language usage informationaccording to web browsing is received.

Referring to FIG. 7 , the processor 170 of the display device 100 maydetermine whether a content-language entity exists in an HTTP response(S701).

The content-language entity may represent the language of a web site.Accordingly, the language of the web site that the user visits whilebrowsing the web may be regarded as the main language of the user.

When it is determined in step S701 that the content-language entityexists in the HTTP response, the processor 170 of the display device 100may determine a language corresponding to the content-language entity asthe main language (S703).

When it is determined in step S701 that the content-language entity doesnot exist in the HTTP response, the processor 170 of the display device100 may determine the language of text included in the web page as themain language (S705).

Even when no content-language entity is present in the HTTP response,the web page may contain text. Accordingly, the processor 170 mayanalyze the language of text included in the web page and determine thecorresponding language as the user’s main language.

When the text included in the web page is composed of a plurality oflanguages, the processor 170 may determine the language representing themost frequency among languages of the text included in the web page asthe user’s main language.

FIG. 8 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

Specifically, FIG. 8 illustrates an embodiment in which the displaydevice 100 determines a main language according to a rule-baseddetermination method in a situation in which language usage informationaccording to output of media content is received.

Referring to FIG. 8 , the processor 170 of the display device 100 maydetermine whether the language of the voice or subtitle of media contentis changed (S801).

In media content, the language of the voice is basically fixed, but insome cases, the language of the voice may be selected from among severalcandidate languages. Therefore, when the language of voice of the mediacontent is changed, it can be seen that the user prefers the language ofthe voice after the change to the language of the voice before thechange.

However, since the number of languages of the voice provided by themedia content is insufficient compared to the number of the languages ofthe subtitle provided by the media content, it may be difficult to seethat the voice language represents the user’s main language.Accordingly, it can be seen that the language of the subtitle of themedia content has a higher priority than the language of the voice ofthe media content.

When it is determined in step S801 that the language of the voice orsubtitle of the media content is changed, the processor 170 of thedisplay device 100 may determine the changed language of the voice orthe changed language of the subtitle as the main language (S803).

In an embodiment, as described above, the language of the subtitle ofthe media content may have a higher priority than the language of thevoice of the media content. When both the language of the voice and thelanguage of the subtitle are changed, the processor 170 may determinethe changed language of the subtitle as the main language.

When it is determined in step S803 that the language of the voice orsubtitle of the media content is not changed, the processor 170 of thedisplay device 100 may determine the language of the voice or subtitleof the media content as the main language (S805).

In an embodiment, as described above, the language of the subtitle ofthe media content may have a higher priority than the language of thevoice of the media content. When the language of the voice and thelanguage of the subtitle are different from each other, the processor170 may determine the language of the subtitle as the main language.

FIG. 9 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

Specifically, FIG. 9 illustrates an embodiment in which the displaydevice 100 determines a main language according to a rule-baseddetermination method in a situation in which language usage informationaccording to interworking with the user terminal 300 is received.

Referring to FIG. 9 , the processor 170 of the display device 100 maydetermine whether the display device 100 is connected to the userterminal 300 (S901).

The display device 100 and the user terminal 300 may be connectedthrough a pairing process, and the connection between the two devicesmeans that two devices have been currently connected.

When it is determined in step S901 that the display device 100 isconnected to the user terminal 300, the processor 170 of the displaydevice 100 may determine the system language of the connected userterminal 300 as the main language (S903).

When it is determined in step S901 that the display device 100 is notconnected to the user terminal 300, the processor 170 of the displaydevice 100 may determine the system language of the display device 100as the main language (S905).

This may mean that the system language of the user terminal 300 cannotbe determined as the main language because the connected user terminal300 does not exist.

FIGS. 7 to 9 described above show examples of determining the user’smain language by independently considering the user’s each languageusage information, but the present disclosure is not limited thereto. Incontrast, FIG. 10 , which will be described later, shows examples ofdetermining a main language by comprehensively considering variouspieces of language usage information of a user.

FIG. 10 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

Specifically, FIG. 10 illustrates an embodiment in which the displaydevice 100 determines a main language in consideration of a plurality ofpieces of language usage information according to a rule-baseddetermination method.

Referring to FIG. 10 , the processor 170 of the display device 100 mayselect language usage information during a predetermined period fromamong the received pieces of language usage information (S1001).

The predetermined period may mean a period from the present time pointto a certain time point in the past. For example, the predeterminedperiod may mean a period from the current time point to 60 minutesbefore. This means that the recent language usage information has a highthe importance in determining the current main language of the user.

Then, the processor 170 of the display device 100 may assign a weightfor each selected language usage information (S1003).

The processor 170 may assign a weight for each selected language usageinformation in consideration of the type of language usage informationand time information.

The processor 170 may assign a lower weight (or time weight) for eachlanguage usage information as the elapsed time is longer. Conversely,the processor 170 may assign a higher weight for each language usageinformation as the elapsed time is shorter.

The operation of selecting language usage information during apredetermined period in step S1001 may also be viewed as assigning aweight of 0 for language usage information that does not belong withinthe predetermined period, in that the processor 170 assigns a weight forthe language usage information based on the elapsed time. That is, thestep (S1001) of selecting language usage information for thepredetermined period may be viewed as a part of a process of assigning aweight for each language usage information.

The processor 170 may assign a weight (or type weight) for each languageusage information in consideration of the type of language usageinformation. A weight may be preset for each type of language usageinformation. For example, the weight of the content-language entity ofthe web site may be set higher than the weight of the language of textincluded in the web page.

The processor 170 may calculate an overall weight based on a time weightand a type weight assigned for each language usage information. Forexample, the processor 170 may calculate the overall weight bymultiplying the time weight and the type weight assigned for eachlanguage usage information.

Then, the processor 170 of the display device 100 may determine a mainlanguage in consideration of a weight and a priority corresponding toeach selected language usage information (S 1005).

The processor 170 may determine a main language by performing comparisonon pieces of language usage information having the highest priority withrespect to each selected language usage information, and a languagehaving the largest sum of weights (or the sum of scores) may bedetermined as the main language based on the assigned weights withrespect to the pieces of language usage information having the samepriority. For example, when the first language usage information is thefirst priority, the second language usage information is the secondpriority, a language corresponding to the first language usageinformation is English, and a language corresponding to the secondlanguage usage information is Korean, the processor 170 may determineEnglish, which is a language corresponding to the first language usageinformation with the first priority, as the main language. On the otherhand, when both the first language usage information and the secondlanguage usage information have second priority, the languagecorresponding to the first language usage information is English, andthe language corresponding to the second language usage information isKorean, the processor 170 may determine a language having a higherweight among the weight of the first language usage information and theweight of the second language usage information as the main language.Here, the weight may mean an overall weight.

When the processor 170 determines the main language, the priority may beconsidered preferentially compared to the weight. Therefore, even whenthe weight assigned to the first language usage information with thefirst priority, is lower than the weight assigned the second languageusage information with the second priority, the processor 170 maydetermine a language corresponding to the first language usageinformation with the first priority as a main language.

A priority may be preset for each type of language usage information.For example, the system language of the user terminal 300 may have afirst priority, the language of the user’s spoken voice may have asecond priority, and other language usage information may have a thirdpriority.

Each language usage information may be assigned a weight (or typeweight) and a priority based on the type, and the weight and thepriority may be set to have a strong positive correlation. That is, thelanguage usage information with a high priority may be set to have ahigh type weight. Therefore, in another embodiment, the main languagemay be determined by considering only the weight without considering thepriority of each language usage information.

FIG. 11 is a diagram illustrating an example of weights and prioritiesassigned to types of language usage information.

As described above, the weight and the priority may be set to have ahigh positive correlation. Further, according to an embodiment, both theweight and the priority may be considered, or only the weight may beconsidered. On the other hand, only the priority may be consideredaccording to an embodiment.

The weight values to be described below are merely examples, and thevalues may be variously changed according to embodiments.

Referring to FIG. 11 , a weight 1120 and a priority 1130 may be assignedto each language usage information based on the type 1110 of thelanguage usage information.

For example, the system language 1111 of the user terminal 300 may beassigned a first priority, which is the highest priority, and a weightof 1, which is higher than other language usage information. Thelanguage 1112 of a user’s spoken voice may be assigned a second priorityand a weight of 0.5. The language 1113 of subtitle of media content maybe assigned a third priority and a weight of 0.3. The language 1114 ofvoice of media content may be assigned a fifth priority, and a weight of0.1. The content-language entity 1115 of a web site may be assigned athird priority, and a weight of 0.15. The language 1116 of text includedin a web page may be assigned a fourth priority, and a weight of 0.05.

FIG. 12 is an operation flowchart illustrating an example of a step S603of determining a main language based on the language usage informationshown in FIG. 6 .

Specifically, FIG. 12 illustrates an embodiment in which the displaydevice 100 determines a main language in consideration of a plurality ofpieces of language usage information according to a learning-baseddetermination method.

Referring to FIG. 12 , the processor 170 of the display device 100 mayselect language usage information during a predetermined time periodfrom among the received pieces of language usage information (S1201).

The predetermined period may mean a period from the present time pointto a certain time point in the past. For example, the predeterminedperiod may mean a period from the current time point to 60 minutesbefore.

The processor 170 of the display device 100 may determine the mainlanguage from the selected language usage information by using the mainlanguage determination model (S1203).

The main language determination model may refer to an artificialintelligence model that determines and outputs a main languagecorresponding to input language usage information when the languageusage information is input.

In an embodiment, the processor 170 may receive (or collect) the user’slanguage usage information in real time, and determine the main languageby inputting the received language usage information into the mainlanguage determination model. That is, language usage information isinput to the main language determination model in a time series, and themain language determination model may determine a main languagecorresponding to the language usage information input in a time series.To this end, the main language determination model may include aRecurrent Neural Network (RNN).

The main language determination model may be trained based on learningdata labeled with language usage information and a corresponding mainlanguage. When the system language of the display device 100 is changed,or when the change of the system language of the display device 100 istried but the user refuses to change the system language and then thesystem language has not been changed, the learning data used to trainthe main language determination model is generated. For example, whenthe processor 170 automatically changes the system language from thefirst language to the second language in a specific situation, but theuser refuses to change the system language (that is, the system languageis returned from the second language to the first language), theprocessor 170 may generate learning data by labeling a language usagerecord corresponding to the corresponding situation with the firstlanguage.

FIG. 13 is a diagram illustrating an example of a main languagedetermination model according to an embodiment of the presentdisclosure.

Referring to FIG. 13 , a main language determination model 1310 mayinclude an RNN, and when language usage information 1320 is input intime series, output a main language 1330 corresponding to the languageusage information 1320. Since the main language determination model 1310includes the RNN, the main language model 1310 may determine the mainlanguage 1330 based on the values propagated from the main languagedetermination model 1310 at the previous time point as well as thelanguage usage information 1320 at a corresponding time point.

For example, when the language usage information 1320 is input to themain language determination model 1310 at the time point T1, the mainlanguage determination model 1310 may determine a main language 1330 attime point T1 based on input language usage information 1320 and thevalues propagated from the main language determination model 1310 at atime point immediately before T1, and output the determined mainlanguage 1330.

Even when the language usage information 1320 is not input to the mainlanguage determination model 1310 at time point T2, the main languagedetermination model 1310 may determine the main language 1330 at timepoint T2 based on values propagated from the main language determinationmodel 1310 at time point T1, and output the determined main language1330. In this case, since the language usage information 1320 is notinput at time point T2, it is preferable that the main languagedetermination model 1310 is trained such that the main language 1330output at the time point T2 is the same as the main language 1330determined at time point T1.

For example, when the language usage information 1320 is input to themain language determination model 1310 at time point T3, the mainlanguage determination model 1310 may determine the main language 1330at time point T3 based on input language usage information 1320 and thevalues propagated from the main language determination model 1310 attime point T2, and output the determined main language 1330.

FIG. 13 is a diagram illustrating an embodiment in which the mainlanguage determination model 1310 includes an RNN, but the presentdisclosure is not limited thereto. That is, in various embodiments ofthe present disclosure, the main language determination model 1310 maybe composed of an artificial neural network that does not include anRNN.

FIG. 14 is a diagram illustrating an embodiment of the presentdisclosure.

Specifically, FIG. 14 illustrates an embodiment in a case where thelanguage update mode of the display device 1410 is a manual update mode.

Referring to FIG. 14 , when a terminal 1430 of a user 1420 is connectedto a display device 1410, the display device 1410 may determine the mainlanguage of the user 1420 as the system language of the terminal 1430.In addition, the display device 1410 may determine whether the setsystem language and the determined main language are the same. When thesystem language of the display device 1410 is English (EN), the systemlanguage of the terminal 1430 is Korean (KR), and then the two systemlanguages are different from each other, the display device 1410 mayprovide a suggestion of changing the currently set system language‘English (EN)’ to the system language ‘Korean (KR)’ of the connectedterminal.

The display device 1410 may provide the suggestion of changing thesystem language in the manner of at least one of a voice notification1440 and an image notification 1450.

The display device 1410 may provide a suggestion of changing the systemlanguage by outputting the voice notification 1440 of “Do you want tochange the current system language from English (EN) to Korean (KR) thatis the system language of the connected terminal?” or “Do you want tochange the current system language from English (EN) to Korean (KR)?”and determine whether to change the system language based on the voiceresponse of the user 1420 corresponding to the suggestion.

The display device 1410 may provide a suggestion of changing the systemlanguage by outputting the image notification 1450 including a messageof Do you want to change the current system language from English (EN)to Korean (KR) that is the system language of the connected terminal?”or “Do you want to change the current system language from English (EN)to Korean (KR)?”, an acceptance Item 1451, and a refusal Item 1452 anddetermine whether to change the system language based on an input of theuser 1420 corresponding to the suggestion. In this case, the user 1420may apply an input for selecting the acceptance item 1451 or the refusalitem 1452 using the remote control device 200.

When the display device 1410 provides the user 1420 with the suggestionof changing the system language but does not receive a response within apredetermined time, the display device 1410 may determine not to changethe system language.

When the display device 1410 provides the user 1420 with a suggestion tochange the system language, the display device 1410 may generatelearning data including the language usage record of the user 1420 andwhether the system language has been changed, and the generated learningdata may be used to train the main language determination model.

FIG. 15 is a diagram illustrating an embodiment of the presentdisclosure.

Specifically, FIG. 15 illustrates an embodiment in a case where thelanguage update mode of the display device 1410 is an automatic updatemode.

Referring to FIG. 15 , when the terminal 1430 of the user 1420 isconnected to the display device 1410, the display device 1410 maydetermine whether the set system language and the system language of theconnected terminal 1430 are identical to each other. When the systemlanguage of the display device 1410 is English (EN), the system languageof the terminal 1430 is Korean (KR), and then the two system languagesare different from each other, the display device 1410 may change thecurrently set system language from ‘English (EN)’ to ‘Korean (KR)’ thatis the system language of the connected terminal and provide anotification indicating that the system language has been changed (ornotification indicating completion of change of the system language).

Alternatively, the display device 1410 may receive the language usageinformation of the user 1420 including the set system language from theconnected terminal 1430, and determine the main language of the user1420 using the received language usage information and the main languagedetermination model. When it is determined that the main language of theuser 1420 is ‘Korean (KR)’, the display device 1410 may change thecurrently set system language from ‘English (EN)’ to ‘Korean (KR)’ thatis the determined main language. and may provide a notificationindicating that the system language has been changed.

The display device 1410 may provide the notification indicatingcompletion of change of the system language in the manner of at leastone of a voice notification 1540 and an image notification 1550.

The display device 1410 may output a voice notification 1540 of “Thecurrent system language has been changed from the language EN to thelanguage KR” to provide a notification indicating completion of changeof the system language. The display device 1410 may or may not cancelthe operation of changing the system language based on the voiceresponse of the user 1420 after the notification indicating completionof change of the system language is provided.

The display device 1410 may output a message “The current systemlanguage has been changed from the language EN to the language KR” andan image notification 1550 including an OK item 1551 and a cancel item1552 to provide the notification indicating the completion of change ofthe system language and may or may not cancel the operation of changingthe system language based on a corresponding input of the user 1420. Inthis case, the user 1420 may apply an input for selecting the OK item1551 or the cancel item 1552 using the remote control device 200.

When the display device 1410 provides the user 1420 with thenotification indicating the completion of change of the system languagebut does not receive a response within a predetermined time, the displaydevice 1410 may determine not to cancel the operation of changing thesystem language.

When the display device 1410 provides the user 1420 with thenotification indicating the completion of change of the system languageor changes the system language, the display device 1410 may generatelearning data including the language usage record of the user 1420 andwhether the operation of changing the system language is canceled, andthe generated learning data may be used to train the main languagedetermination model.

In the embodiments of FIGS. 14 and 15 , the display device 1410 maydetermine the main language of the user 1420 based on the language usagerecord in the terminal 1430 as well as the system language of theterminal 1430 of the user 1420. The language usage record in theterminal 1430 may include the language of text included in a web pageduring web browsing in the terminal 1430, a content-language entity of aweb site, and the like.

FIG. 16 is a diagram illustrating an embodiment of the presentdisclosure.

Specifically, FIG. 16 illustrates an embodiment in a case where thelanguage update mode of the display device 1410 is a manual update mode.

Referring to FIG. 16 , a remote control device 1630 may be connected toa display device 1610, and the display device 1610 may be controlled bythe remote control device 1630. When the user 1620 performs control tochange the language of the subtitle of media content of the displaydevice 1610 through the remote control device 1630, the display device1610 may determine the language of the subtitle of the media content asthe main language of the user 1620. In addition, the display device 1610may determine whether the set system language and the determined mainlanguage are the same. When the system language of the display device1610 is English (EN), the language of the voice 1611 of the mediacontent provided by the display device 1610 is English (EN), and thelanguage of the subtitle 1612 of the media content set by the user 1620is Korean (KR), the display device 1610 may provide a system languagechange suggestion of changing the currently set system language ‘English(EN)’ to the language ‘Korean (KR)’ of the subtitle 1612 of the mediacontent.

The display device 1610 may provide the suggestion of changing thesystem language in the manner of at least one of a voice notification1640 and an image notification 1650.

The display device 1610 may provide a suggestion of changing the systemlanguage by outputting the voice notification 1640 of “Do you want tochange the current system language from the language EN to the languageKR ?” and determine whether to change the system language based on thevoice response of the user 1620 corresponding to the suggestion.

The display device 1610 may provide a suggestion of changing the systemlanguage by outputting the image notification 1650 including a messageof “Do you want to change the current system language from the languageEN to the language KR?”, an acceptance Item 1651, and a refusal Item1652 and determine whether to change the system language based on aninput of the user 1620 corresponding to the suggestion. In this case,the user 1620 may apply an input for selecting the acceptance item 1651or the refusal item 1652 using the remote control device 200.

When the display device 1610 provides the user 1620 with the suggestionof changing the system language but does not receive a response within apredetermined time, the display device 1410 may determine not to changethe system language.

When the display device 1610 provides the user 1620 with a suggestion tochange the system language, the display device 1610 may generatelearning data including the language usage record of the user 1620 andwhether the system language has been changed, and the generated learningdata may be used to train the main language determination model.

FIG. 17 shows an embodiment of the present disclosure.

Referring to FIG. 17 , a remote control device 1630 may be connected toa display device 1610, and the display device 1610 may be controlled bythe remote control device 1630. When the user 1620 performs control tochange the language of the subtitle of media content of the displaydevice 1610 through the remote control device 1630, the display device1610 may determine the language of the subtitle of the media content asthe main language of the user 1620. In addition, the display device 1610may determine whether the set system language and the determined mainlanguage are the same. When the system language of the display device1610 is English (EN), the language of the voice 1611 of the mediacontent provided by the display device 1610 is English (EN), and thelanguage of the subtitle 1612 of the media content set by the user 1620is Korean (KR), the display device 1610 may change the currently setsystem language from ‘English (EN)’ to the language ‘Korean (KR)’ of thesubtitle 1612 of the media content and provide a notification indicatingthat the system language has been changed.

Alternatively, the display device 1610 may receive the language usageinformation of the user 1620 including the language of subtitle 1612 ofmedia content changed by the control of the connected remote controldevice 1630, and determine the main language of the user 1620 using thereceived language usage information and the main language determinationmodel. When it is determined that the main language of the user 1620 is‘Korean (KR)’, the display device 1610 may change the currently setsystem language from ‘English (EN)’ to ‘Korean (KR)’ that is thedetermined main language. and may provide a notification indicating thatthe system language has been changed.

The display device 1610 may provide the notification indicatingcompletion of change of the system language in the manner of at leastone of a voice notification 1740 and an image notification 1750.

The display device 1610 may output a voice notification 1740 of “Thecurrent system language has been changed from the language EN to thelanguage KR” to provide a notification indicating completion of changeof the system language. The display device 1610 may or may not cancelthe operation of changing the system language based on the voiceresponse of the user 1620 after the notification indicating completionof change of the system language is provided.

The display device 1610 may output a message “The current systemlanguage (EN) has been changed from the language EN to the language KR”and an image notification 1750 including an OK item 1751 and a refusalitem 1752 to provide the notification indicating the completion ofchange of the system language and may or may not cancel the operation ofchanging the system language based on a corresponding input of the user1620. In this case, the user 1620 may apply an input for selecting theOK item 1751 or the cancel item 1752 using the remote control device200.

When the display device 1610 provides the user 1620 with thenotification indicating the completion of change of the system languagebut does not receive a response within a predetermined time, the displaydevice 1410 may determine not to cancel the operation of changing thesystem language.

When the display device 1610 provides the user 1620 with thenotification indicating the completion of change of the system languageor changes the system language, the display device 1610 may generatelearning data including the language usage record of the user 1620 andwhether the operation of changing the system language is canceled, andthe generated learning data may be used to train the main languagedetermination model.

Although FIGS. 16 and 17 illustrate examples in which the language ofthe voice 1611 of the media content is different from the language ofthe subtitle 1612 of the media content, the present disclosure is notlimited thereto. That is, in various embodiments of the presentdisclosure, even when the language of the subtitle 1612 of the mediacontent is set to be the same as the language of the voice 1611 of themedia content, the language of the subtitle 1612 of the media contentmay be determined as the main language.

According to an embodiment of the present disclosure, theabove-described method may be implemented with codes readable by acomputer on a medium in which a program is recorded. Thecomputer-readable medium includes all kinds of recording devices inwhich data readable by a computer system is stored. Examples of thecomputer readable medium may include a hard disk drive (HDD), a solidstate disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, amagnetic tape, a floppy disk, an optical data storage device, and thelike.

1. A display device comprising: a display; a communication interfaceconfigured to communicate with an external device; and a processorconfigured to receive language usage information of a user, determine amain language of the user based on the language usage information,compare a set system language with the main language, and when thesystem language is different from the main language, change the systemlanguage such that the system language is set to the main language orperform suggestion that the system language is set to the main language.2. The display device of claim 1, wherein the language usage informationincludes at least one of a content-language entity of an HTTP responsereceived from a website, language information of text included in a webpage, language information of voice of media content, languageinformation of subtitles of the media content, language information ofthe user’s spoken voice, and system language information of a userterminal.
 3. The display device of claim 2, wherein the processor isconfigured to select language usage information for a predeterminedperiod from among the language usage information, assign a weight toeach of the selected language usage information, and determine the mainlanguage in consideration of the weight corresponding to each of theselected language usage information.
 4. The display device of claim 3,wherein the processor is configured to assign a higher time weight toeach of the selected language usage information as an elapsed time isshorter, assign a predetermined type weight to each of the selectedlanguage usage information for each type of language usage information,and calculate an overall weight based on the time weight and the typeweight.
 5. The display device of claim 4, wherein the processor isconfigured to calculate a sum of the overall weights for each languagebased on the overall weight calculated for each of the selected languageusage information, and determine a language having a largest sum of theoverall weights as the main language.
 6. The display device of claim 3,wherein the processor is configured to determine the main language inconsideration of a weight corresponding to the selected language usageinformation and a predetermined priority for each type of language usageinformation.
 7. The display device of claim 6, wherein the processor isconfigured to select a first language usage information having a highestpriority from among the selected language usage information, anddetermine the main language in consideration of a weight of the firstlanguage usage information.
 8. The display device of claim 6, wherein apriority of the system language of the user terminal is higher than apriority of another language usage information, and a priority of thelanguage of the subtitles of the media content is higher than a priorityof the language of the voice of the media content.
 9. The display deviceof claim 3, wherein the processor is configured to select language usageinformation for a predetermined period from among the language usageinformation, and determine the main language using the selected languageusage information and a main language determination model, and whereinthe main language determination model is a model including an artificialneural network (ANN), the model being trained based on a machinelearning algorithm or a deep learning algorithm.
 10. The display deviceof claim 9, wherein the main language determination model is a modelthat includes a Recurrent Neural Network (RNN), and determines andoutputs the main language when the language usage information is inputin real time.
 11. The display device of claim 9, wherein the processoris configured to generate learning data corresponding to information ona used language and a result of change when changing or attempting tochange the system language, and wherein the learning data is used toupdate the main language determination model.
 12. The display device ofclaim 1, wherein the processor is configured to automatically change thesystem language such that the system language is set to the mainlanguage when the system language and the main language are differentfrom each other, output a first notification indicating that the systemlanguage has been changed, and determine whether to cancel change of thesystem language based on an input of the user corresponding to the firstnotification in a case where a language update mode is an automaticupdate mode.
 13. The display device of claim 12, wherein the processoris configured to output a second notification suggesting change of thesystem language such that the system language is set to the mainlanguage when the system language and the main language are differentfrom each other, output a first notification indicating that the systemlanguage has been changed, and determine whether to change the systemlanguage based on an input of the user corresponding to the secondnotification in a case where the language update mode is a manual updatemode.
 14. A method for operating a display device comprising: receivinglanguage usage information of a user; determining a main language of theuser based on the language usage information; comparing a set systemlanguage with the main language; and changing the system language suchthat the system language is set to the main language or performingsuggestion that the system language is set to the main language, whenthe system language is different from the main language.
 15. A recordingmedium recording a method of operating a display device, the methodcomprises: receiving language usage information of a user; determining amain language of the user based on the language usage information;comparing a set system language with the main language; and changing thesystem language such that the system language is set to the mainlanguage or performing suggestion that the system language is set to themain language, when the system language is different from the mainlanguage.